[draft] Preserve underscores in custom schema name prefixes by lennartkats-db · Pull Request #7287 · databricks/cli

lennartkats-db · 2026-11-02T11:37:45Z

Changes

Preserve literal trailing and repeated underscores from custom presets.name_prefix values in schema, registered model, and model serving endpoint names. Other special characters still normalize as before, and generated development prefixes retain their separator.

Why

Issue #2305 reports that ${workspace.current_user.short_name}_ joined with schema qc produces my_userqc instead of my_user_qc. A follow-up reports the same behavior for registered models and serving endpoints.

Existing deployments: Correcting these effective names can change resource identities after a CLI upgrade, even when the bundle configuration has not changed. With the current direct engine, affected schemas rename in place and keep their schema_id, but references to the old qualified name need updating. Registered models and serving endpoints still plan recreation when their names change. Existing users should inspect bundle plan and plan updates to references and endpoint clients before deploying.

The issue discussion flags compatibility concerns. Maintainers need to decide whether changing the default normalization is acceptable for a release or whether the correction should require an opt-in setting. This PR currently changes the default.

Fixes #2305

Tests

go test ./bundle/config/mutator/resourcemutator -run '^TestApplyPresets' -count=1
go test ./acceptance -run '^TestAccept/bundle/presets/(name_prefix_underscores|name_prefix_existing_schema)$' -count=1: both local DMS variants. The existing-state fixture simulates an old effective schema name, confirms an in-place rename to the corrected name with the same schema_id, and then checks for a clean plan.
./task checks (including changelog validation). The new YAML was formatted with the repository's yamlfmt tool.

Remote migration was not exercised.

This PR was written by Codex.

eng-dev-ecosystem-bot · 2026-11-02T11:53:01Z

✅ Integration test report

Commit: d8674d9

Run: 43449699151

	Env	🔄flaky	💚RECOVERED	✅pass	🙈skip	Time
💚	aws linux-2core-8gb		1	270	17	12:23
💚	aws-windows-latest-4core-16gb		1	272	15	8:05
💚	azure linux-2core-8gb		1	269	17	10:29
💚	azure-windows-latest-4core-16gb		1	271	15	8:49
🔄	gcp linux-2core-8gb	1	1	269	17	12:04
💚	gcp-windows-latest-4core-16gb		1	272	15	8:20

	Test Name	aws linux-2core-8gb	aws-windows-latest-4core-16gb	azure linux-2core-8gb	azure-windows-latest-4core-16gb	gcp linux-2core-8gb	gcp-windows-latest-4core-16gb
💚	TestAccept	💚R	💚R	💚R	💚R	💚R	💚R
🔄	TestSyncFullFileSync	✅p	✅p	✅p	✅p	🔄f	✅p

Top 9 slowest tests (at least 2 minutes):

duration	env	testname
3:19	gcp-windows-latest-4core-16gb	TestAccept
3:09	azure-windows-latest-4core-16gb	TestAccept
3:09	aws-windows-latest-4core-16gb	TestAccept
2:56	aws linux-2core-8gb	TestFilerWorkspaceFilesExtensionsReadDir
2:32	aws linux-2core-8gb	TestExportDir
2:24	aws-windows-latest-4core-16gb	TestFilerWorkspaceFilesExtensionsRead
2:19	azure-windows-latest-4core-16gb	TestFilerRecursiveDelete/workspace_files
2:18	azure-windows-latest-4core-16gb	TestFilerWorkspaceFilesExtensionsDelete
2:03	gcp-windows-latest-4core-16gb	TestFsCpDirToDirFileNotOverwritten/dbfs_to_uc-volumes

github-actions · 2026-11-13T10:54:12Z

Commit: d8674d9c7 | Workflow run

Regression test report

Selected 2 changed tests (limit=20, 0 not selected)

4 tests: regression=4 unreleased=0 coverage=0 failing=0

test	branch	main (`8e70c8b`)	latest (v1.26.0)
TestAccept/bundle/presets/name_prefix_existing_schema/DMS=	✅	❌	❌
TestAccept/bundle/presets/name_prefix_existing_schema/DMS=true	✅	❌	❌
TestAccept/bundle/presets/name_prefix_underscores/DMS=	✅	❌	❌
TestAccept/bundle/presets/name_prefix_underscores/DMS=true	✅	❌	❌

TestAccept/bundle/presets/name_prefix_existing_schema/DMS=: branch ✅ | main (8e70c8b) ❌ | latest (v1.26.0) ❌

main (8e70c8b):

acceptance_test.go:1374: Diff:
        --- bundle/presets/name_prefix_existing_schema/output.txt
        +++ /tmp/TestAcceptbundlepresetsname_prefix_existing_schemaDMS=1635206995/002/output.txt
        @@ -16,21 +16,14 @@
         >>> update_file.py databricks.yml name_prefix: my_user name_prefix: my_user_
         
         >>> [CLI] bundle plan
        -update schemas.qc
        +Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
         
        -Plan: 0 to add, 1 to change, 0 to delete, 0 unchanged
        -
         >>> [CLI] bundle deploy
         Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/name-prefix-existing-schema/existing/files...
        -Updated schemas.qc
         Files: 1 uploaded, 0 deleted
        -Resources: 0 created, 1 changed, 0 deleted, 0 unchanged
        +Resources: 0 created, 0 changed, 0 deleted, 1 unchanged
         
         >>> [CLI] schemas get main.my_user_qc
        -{
        -  "full_name": "main.my_user_qc",
        -  "schema_id": "[SCHEMA_ID]"
        -}
        +Error: Schema 'main.my_user_qc' does not exist.
         
        ->>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
        +Exit code: 1
        
    acceptance_test.go:1164: Test produced unexpected files:
        .databricks/.gitignore
        .databricks/bundle/existing/deployment.json
        .databricks/bundle/existing/resources.json
        .databricks/bundle/existing/sync-snapshots/4be0900586ff7b37.json

latest (v1.26.0):

acceptance_test.go:1374: Diff:
        --- bundle/presets/name_prefix_existing_schema/output.txt
        +++ /tmp/TestAcceptbundlepresetsname_prefix_existing_schemaDMS=130693523/002/output.txt
        @@ -16,21 +16,14 @@
         >>> update_file.py databricks.yml name_prefix: my_user name_prefix: my_user_
         
         >>> [CLI] bundle plan
        -update schemas.qc
        +Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
         
        -Plan: 0 to add, 1 to change, 0 to delete, 0 unchanged
        -
         >>> [CLI] bundle deploy
         Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/name-prefix-existing-schema/existing/files...
        -Updated schemas.qc
         Files: 1 uploaded, 0 deleted
        -Resources: 0 created, 1 changed, 0 deleted, 0 unchanged
        +Resources: 0 created, 0 changed, 0 deleted, 1 unchanged
         
         >>> [CLI] schemas get main.my_user_qc
        -{
        -  "full_name": "main.my_user_qc",
        -  "schema_id": "[SCHEMA_ID]"
        -}
        +Error: Schema 'main.my_user_qc' does not exist.
         
        ->>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
        +Exit code: 1
        
    acceptance_test.go:1164: Test produced unexpected files:
        .databricks/.gitignore
        .databricks/bundle/existing/deployment.json
        .databricks/bundle/existing/resources.json
        .databricks/bundle/existing/sync-snapshots/7b47dfd882ac6b81.json

TestAccept/bundle/presets/name_prefix_existing_schema/DMS=true: branch ✅ | main (8e70c8b) ❌ | latest (v1.26.0) ❌

main (8e70c8b):

acceptance_test.go:1374: Diff:
        --- bundle/presets/name_prefix_existing_schema/output.txt
        +++ /tmp/TestAcceptbundlepresetsname_prefix_existing_schemaDMS=true2613543345/002/output.txt
        @@ -16,21 +16,14 @@
         >>> update_file.py databricks.yml name_prefix: my_user name_prefix: my_user_
         
         >>> [CLI] bundle plan
        -update schemas.qc
        +Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
         
        -Plan: 0 to add, 1 to change, 0 to delete, 0 unchanged
        -
         >>> [CLI] bundle deploy
         Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/name-prefix-existing-schema/existing/files...
        -Updated schemas.qc
         Files: 1 uploaded, 0 deleted
        -Resources: 0 created, 1 changed, 0 deleted, 0 unchanged
        +Resources: 0 created, 0 changed, 0 deleted, 1 unchanged
         
         >>> [CLI] schemas get main.my_user_qc
        -{
        -  "full_name": "main.my_user_qc",
        -  "schema_id": "[SCHEMA_ID]"
        -}
        +Error: Schema 'main.my_user_qc' does not exist.
         
        ->>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
        +Exit code: 1
        
    acceptance_test.go:1164: Test produced unexpected files:
        .databricks/.gitignore
        .databricks/bundle/existing/deployment.json
        .databricks/bundle/existing/resources.json
        .databricks/bundle/existing/sync-snapshots/4be0900586ff7b37.json

latest (v1.26.0):

acceptance_test.go:1374: Diff:
        --- bundle/presets/name_prefix_existing_schema/output.txt
        +++ /tmp/TestAcceptbundlepresetsname_prefix_existing_schemaDMS=true3443186218/002/output.txt
        @@ -16,21 +16,14 @@
         >>> update_file.py databricks.yml name_prefix: my_user name_prefix: my_user_
         
         >>> [CLI] bundle plan
        -update schemas.qc
        +Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
         
        -Plan: 0 to add, 1 to change, 0 to delete, 0 unchanged
        -
         >>> [CLI] bundle deploy
         Uploading bundle files to /Workspace/Users/[USERNAME]/.bundle/name-prefix-existing-schema/existing/files...
        -Updated schemas.qc
         Files: 1 uploaded, 0 deleted
        -Resources: 0 created, 1 changed, 0 deleted, 0 unchanged
        +Resources: 0 created, 0 changed, 0 deleted, 1 unchanged
         
         >>> [CLI] schemas get main.my_user_qc
        -{
        -  "full_name": "main.my_user_qc",
        -  "schema_id": "[SCHEMA_ID]"
        -}
        +Error: Schema 'main.my_user_qc' does not exist.
         
        ->>> [CLI] bundle plan
        -Plan: 0 to add, 0 to change, 0 to delete, 1 unchanged
        +Exit code: 1
        
    acceptance_test.go:1164: Test produced unexpected files:
        .databricks/.gitignore
        .databricks/bundle/existing/deployment.json
        .databricks/bundle/existing/resources.json
        .databricks/bundle/existing/sync-snapshots/7b47dfd882ac6b81.json
FAIL

TestAccept/bundle/presets/name_prefix_underscores/DMS=: branch ✅ | main (8e70c8b) ❌ | latest (v1.26.0) ❌

main (8e70c8b):

acceptance_test.go:1374: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1374
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1115
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:678
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle validate -o json -t trailing\n{\n  \"schema\": \"[USERNAME]_qc\",\n  \"model\": \"[USERNAME]_model\",\n  \"endpoint\": \"[USERNAME]_endpoint\"\n}\n\n>>> [CLI] bundle validate -o json -t repeated\n{\n  \"schema\": \"team__[USERNAME]__qc\",\n  \"model\": \"team__[USERNAME]__model\",\n  \"endpoint\": \"team__[USERNAME]__endpoint\"\n}\n\n>>> [CLI] bundle validate -o json -t generated\n{\n  \"schema\": \"dev_[USERNAME]_qc\",\n  \"model\": \"dev_[USERNAME]_model\",\n  \"endpoint\": \"dev_[USERNAME]_endpoint\"\n}\n"
        	            	actual  : "\n>>> [CLI] bundle validate -o json -t trailing\n{\n  \"schema\": \"[USERNAME]qc\",\n  \"model\": \"[USERNAME]model\",\n  \"endpoint\": \"[USERNAME]endpoint\"\n}\n\n>>> [CLI] bundle validate -o json -t repeated\n{\n  \"schema\": \"team_[USERNAME]qc\",\n  \"model\": \"team_[USERNAME]model\",\n  \"endpoint\": \"team_[USERNAME]endpoint\"\n}\n\n>>> [CLI] bundle validate -o json -t generated\n{\n  \"schema\": \"dev_[USERNAME]_qc\",\n  \"model\": \"dev_[USERNAME]_model\",\n  \"endpoint\": \"dev_[USERNAME]_endpoint\"\n}\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -3,5 +3,5 @@
        	            	 {
        	            	-  "schema": "[USERNAME]_qc",
        	            	-  "model": "[USERNAME]_model",
        	            	-  "endpoint": "[USERNAME]_endpoint"
        	            	+  "schema": "[USERNAME]qc",
        	            	+  "model": "[USERNAME]model",
        	            	+  "endpoint": "[USERNAME]endpoint"
        	            	 }
        	            	@@ -10,5 +10,5 @@
        	            	 {
        	            	-  "schema": "team__[USERNAME]__qc",
        	            	-  "model": "team__[USERNAME]__model",
        	            	-  "endpoint": "team__[USERNAME]__endpoint"
        	            	+  "schema": "team_[USERNAME]qc",
        	            	+  "model": "team_[USERNAME]model",
        	            	+  "endpoint": "team_[USERNAME]endpoint"
        	            	 }
        	Test:       	TestAccept/bundle/presets/name_prefix_underscores/DMS=
        	Messages:   	bundle/presets/name_prefix_underscores/output.txt vs /tmp/TestAcceptbundlepresetsname_prefix_underscoresDMS=1229222528/002/output.txt

latest (v1.26.0): same output as main (8e70c8b)

TestAccept/bundle/presets/name_prefix_underscores/DMS=true: branch ✅ | main (8e70c8b) ❌ | latest (v1.26.0) ❌

main (8e70c8b):

acceptance_test.go:1374: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1374
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1115
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:678
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle validate -o json -t trailing\n{\n  \"schema\": \"[USERNAME]_qc\",\n  \"model\": \"[USERNAME]_model\",\n  \"endpoint\": \"[USERNAME]_endpoint\"\n}\n\n>>> [CLI] bundle validate -o json -t repeated\n{\n  \"schema\": \"team__[USERNAME]__qc\",\n  \"model\": \"team__[USERNAME]__model\",\n  \"endpoint\": \"team__[USERNAME]__endpoint\"\n}\n\n>>> [CLI] bundle validate -o json -t generated\n{\n  \"schema\": \"dev_[USERNAME]_qc\",\n  \"model\": \"dev_[USERNAME]_model\",\n  \"endpoint\": \"dev_[USERNAME]_endpoint\"\n}\n"
        	            	actual  : "\n>>> [CLI] bundle validate -o json -t trailing\n{\n  \"schema\": \"[USERNAME]qc\",\n  \"model\": \"[USERNAME]model\",\n  \"endpoint\": \"[USERNAME]endpoint\"\n}\n\n>>> [CLI] bundle validate -o json -t repeated\n{\n  \"schema\": \"team_[USERNAME]qc\",\n  \"model\": \"team_[USERNAME]model\",\n  \"endpoint\": \"team_[USERNAME]endpoint\"\n}\n\n>>> [CLI] bundle validate -o json -t generated\n{\n  \"schema\": \"dev_[USERNAME]_qc\",\n  \"model\": \"dev_[USERNAME]_model\",\n  \"endpoint\": \"dev_[USERNAME]_endpoint\"\n}\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -3,5 +3,5 @@
        	            	 {
        	            	-  "schema": "[USERNAME]_qc",
        	            	-  "model": "[USERNAME]_model",
        	            	-  "endpoint": "[USERNAME]_endpoint"
        	            	+  "schema": "[USERNAME]qc",
        	            	+  "model": "[USERNAME]model",
        	            	+  "endpoint": "[USERNAME]endpoint"
        	            	 }
        	            	@@ -10,5 +10,5 @@
        	            	 {
        	            	-  "schema": "team__[USERNAME]__qc",
        	            	-  "model": "team__[USERNAME]__model",
        	            	-  "endpoint": "team__[USERNAME]__endpoint"
        	            	+  "schema": "team_[USERNAME]qc",
        	            	+  "model": "team_[USERNAME]model",
        	            	+  "endpoint": "team_[USERNAME]endpoint"
        	            	 }
        	Test:       	TestAccept/bundle/presets/name_prefix_underscores/DMS=true
        	Messages:   	bundle/presets/name_prefix_underscores/output.txt vs /tmp/TestAcceptbundlepresetsname_prefix_underscoresDMS=true4134842565/002/output.txt
FAIL

latest (v1.26.0):

acceptance_test.go:1374: 
        	Error Trace:	/home/runner/work/cli/cli/libs/testdiff/testdiff.go:22
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1374
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:1115
        	            				/home/runner/work/cli/cli/acceptance/acceptance_test.go:678
        	Error:      	Not equal: 
        	            	expected: "\n>>> [CLI] bundle validate -o json -t trailing\n{\n  \"schema\": \"[USERNAME]_qc\",\n  \"model\": \"[USERNAME]_model\",\n  \"endpoint\": \"[USERNAME]_endpoint\"\n}\n\n>>> [CLI] bundle validate -o json -t repeated\n{\n  \"schema\": \"team__[USERNAME]__qc\",\n  \"model\": \"team__[USERNAME]__model\",\n  \"endpoint\": \"team__[USERNAME]__endpoint\"\n}\n\n>>> [CLI] bundle validate -o json -t generated\n{\n  \"schema\": \"dev_[USERNAME]_qc\",\n  \"model\": \"dev_[USERNAME]_model\",\n  \"endpoint\": \"dev_[USERNAME]_endpoint\"\n}\n"
        	            	actual  : "\n>>> [CLI] bundle validate -o json -t trailing\n{\n  \"schema\": \"[USERNAME]qc\",\n  \"model\": \"[USERNAME]model\",\n  \"endpoint\": \"[USERNAME]endpoint\"\n}\n\n>>> [CLI] bundle validate -o json -t repeated\n{\n  \"schema\": \"team_[USERNAME]qc\",\n  \"model\": \"team_[USERNAME]model\",\n  \"endpoint\": \"team_[USERNAME]endpoint\"\n}\n\n>>> [CLI] bundle validate -o json -t generated\n{\n  \"schema\": \"dev_[USERNAME]_qc\",\n  \"model\": \"dev_[USERNAME]_model\",\n  \"endpoint\": \"dev_[USERNAME]_endpoint\"\n}\n"
        	            	
        	            	Diff:
        	            	--- Expected
        	            	+++ Actual
        	            	@@ -3,5 +3,5 @@
        	            	 {
        	            	-  "schema": "[USERNAME]_qc",
        	            	-  "model": "[USERNAME]_model",
        	            	-  "endpoint": "[USERNAME]_endpoint"
        	            	+  "schema": "[USERNAME]qc",
        	            	+  "model": "[USERNAME]model",
        	            	+  "endpoint": "[USERNAME]endpoint"
        	            	 }
        	            	@@ -10,5 +10,5 @@
        	            	 {
        	            	-  "schema": "team__[USERNAME]__qc",
        	            	-  "model": "team__[USERNAME]__model",
        	            	-  "endpoint": "team__[USERNAME]__endpoint"
        	            	+  "schema": "team_[USERNAME]qc",
        	            	+  "model": "team_[USERNAME]model",
        	            	+  "endpoint": "team_[USERNAME]endpoint"
        	            	 }
        	Test:       	TestAccept/bundle/presets/name_prefix_underscores/DMS=true
        	Messages:   	bundle/presets/name_prefix_underscores/output.txt vs /tmp/TestAcceptbundlepresetsname_prefix_underscoresDMS=true1793890030/002/output.txt

…/dabs/d7af7f468c3a

[factory] Fix qualified DABs behavior

6692d77

lennartkats-db requested review from a team as code owners November 2, 2026 11:37

lennartkats-db requested review from rclarey and shreyas-goenka November 2, 2026 11:37

lennartkats-db added DABs DABs related issues software-factory PR produced by the software factories task labels Nov 2, 2026

[factory] Repair attached DABs CI

f04aa2d

lennartkats-db added software-factory PR produced by the software factories task and removed software-factory PR produced by the software factories task labels Nov 3, 2026

lennartkats-db changed the title ~~[draft] Fix bundle/config/mutator/resourcemutator/apply_presets.go~~ Preserve underscores in custom schema name prefixes Nov 4, 2026

lennartkats-db changed the title ~~Preserve underscores in custom schema name prefixes~~ [draft] Preserve underscores in custom schema name prefixes Nov 4, 2026

Preserve underscores in normalized bundle preset names

507953d

lennartkats-db added the ai-auto PRs made by AI loops & factories, label Nov 13, 2026

lennartkats-db added 3 commits November 13, 2026 14:31

Document schema prefix migration risk

83c05ec

Merge branch 'main' of https://github.com/databricks/cli into factory…

342ca22

…/dabs/d7af7f468c3a

Align prefix migration evidence with schema rename

d8674d9

Navigation Menu

Repository navigation

[draft] Preserve underscores in custom schema name prefixes - #7287

[draft] Preserve underscores in custom schema name prefixes#7287
lennartkats-db wants to merge 6 commits into
mainfrom
factory/dabs/d7af7f468c3a

lennartkats-db commented Nov 2, 2026 •

edited

Loading

Uh oh!

eng-dev-ecosystem-bot commented Nov 2, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Nov 13, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

lennartkats-db commented Nov 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Why

Tests

Uh oh!

eng-dev-ecosystem-bot commented Nov 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Integration test report

Uh oh!

github-actions Bot commented Nov 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Regression test report

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

lennartkats-db commented Nov 2, 2026 •

edited

Loading

eng-dev-ecosystem-bot commented Nov 2, 2026 •

edited

Loading

github-actions Bot commented Nov 13, 2026 •

edited

Loading